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MANAGEMENT  WITHOUT  A  "BOTTOM  LINE" 


I.  MARKET  AND  NON  MARKET  ENTERPRISES 


Enterprises  and  organizations  can  be  categorized  in  a  number 
of  ways:  public  versus  private,  and  government  versus  non-government; 
profit  versus  nonprofit;  regulated  versus  unregulated;  and  market 
versus  non-market.  Each  set  of  categories  has  some  merit  and  utility, 
highlighting  some  distinguishing  characteristics  that  affect,  or 
allegedly  affect,  behavior  of  the  enterprise. 

The  various  categories  often  cut  across  one  another's  purview. 

For  example,  public  enterprises  may  be  profit  makers,  such  as  the 
Export-Import  Bank  and  the  World  Bank,  and  private  corporations 
may  be  non-profit  organizations,  such  as  The  Rand  Corporation. 

The  distinction  between  market  enterprises  and  non-market 
enterprises  has  seemed  to  me  of  particular  interest  and  importance, 
for  reasons  that  I  have  dealt  with  at  length  elsewhere.  Non- 
market  enterprises  are  those  whose  principal  revenues  come  from 
taxes,  donations,  contributions,  endowments,  or  sources  other  than 
the  proceeds  derived  by  charging  prices  for  products  or  services  sold 


Prepared  for  a  Symposium  on  "Governance  of  Public  Enterprises," 
organized  by  the  Minnesota  Law  Review  and  scheduled  for  publication 
in  the  November  1982  issue  of  that  journal. 

For  a  more  extensive  treatment,  on  which  this  discussion  is 
based,  see  my  "A  Theory  of  Non-Market  Failure:  Framework  for 
Implementation  Analysis,"  The  Journal  of  Law  and  Economics,  April  1979, 
and  the  further  discussion  in  "'Non-Market  Failure'  Revisited: 

The  Anatomy  and  Physiology  of  Government  Deficiencies,"  The  Rand 
Corporation,  March  1981. 


to  buyers  who  are  able  to  buy  or  not  to  buy  as  they  wish.  Market 
enterprises  are  those  whose  principal  revenue  comes  from  these 
latter  sources.  In  this  terminology,  "public"  enterprise  is 
mixed  in  character:  public  education,  the  police,  the  National 
Science  Foundation,  and  the  Postal  Service  are  non-market  enterprises. 
COMSAT,  the  Los  Angeles  Department  of  Water  and  Power,  the  railroads 
in  most  European  countries  (where  they  are  operated  by  government) 
are  market  enterprises.  "Public"  or  government  enterprises  that 
operate  in  functioning  markets  are  market  enterprises  if  the  bulk 
of  their  revenues  is  earned  from  prices  charged  to  consumers  for 
transactions  occurring  in  these  markets.  Conversely,  private 
enterprises  whose  revenues  come  principally  from  non-price  sources, 
e.g. ,  endowment  earnings,  government  subsidies  or  appropriations, 
are  non-market  enterprises. 

One  of  the  common,  as  well  as  crucial,  characteristics  of 
non-market  enterprises  (NMEs)  is  that  they  operate  without  the  types 
of  valuable  information  determining  the  "bottom  line"  of  market 
enterprises.  The  profit-and-loss  bottom  line  provides  for  the  market 
enterprise  (ME)  chief  executives,  shareholders  and  the  general 
public  a  quintessential  indicator  of  how  well  the  ME  is  performing. 
This  point  remains  valid,  notwithstanding  that:  (1)  the  P&L 
indicator  may  be  misleading  because  of  distortions  in  it  introduced 
by  inflation,  by  inappropriate  depreciation  allowances,  or  by  special 

* 

The  Postal  Service  is  perhaps  in  transition  because  its  services 
are  increasingly  subject  to  direct  competition  by  market  organizations 
and  Congress  is  showing  less  willingness  to  finance  its  operating 
deficit. 


accounting  practices  used  in  evaluating  inventories;  and  (2)  large 
MEs  are  increasingly  concerned  with  broader  social  and  public  aspects 
of  their  performance  than  those  which  are  directly  reflected  in  the 
P&L.  For  example,  top  management  of  large  MEs  is  acutely  concerned  with 
the  present  status,  as  well  as  prospects,  of  regulation  in  their  industries, 
the  prospective  as  well  as  current  tax  burdens  they  will  be  facing,  the 
state  of  technology  and  its  prospects  for  major  change  in  their  industries, 
and  with  prevailing  public  attitudes  toward  their  firms. 

However,  even  these  "broader"  concerns  relate,  in  a  qualitative 
way,  to  P&L  statements  that  are  downstream.  In  effect,  most  of  these 
larger  concerns  involve  an  implicit  tradeoff  between  P&L  today  and 
P&L  tomorrow. 

In  the  case  of  public  MEs,  the  types  of  "broader"  concerns  that 
are  not  directly  reflected  in  today's  P&L  are  likely  to  be  given 
particular  weight  by  top  management,  because  these  concerns  reflect  the 
major  public  purposes  that  ostensibly  provide  the  rationale  for 
public  ownership  and  operation,  in  the  first  instance.  Stated  another 
way,  public  MEs  are  likely  to  give  greater  weight  to  factors  affecting 
future  P&Ls  rather  than  today's  P&L,  compared  with  the  weight  accorded  them 
by  private  MEs,  i.e.,  public  MEs  will  tend  to  have  a  lower  time  discount 


than  do  private  MEs.  Also,  public  MEs  are  likely  to  be  more  influenced 
by  considerations  affecting  dimensions  of  performance  that  don't 

/£*•  J 

enter  the  P&L  at  all.  For  example,  public  MEs  are  likely  to  be  / 1. 

JX Sc* 

concerned  with  how  to  distribute  profits,  or  how  to  limit  them  t hroughf^^^j^yj 
suitable  pricing  policies,  in  accord  with  some  principles  of  ethical 
and  political  equity  rather  than  shareholder  equity,  and  with  how 


and  political  equity  rather  than  shareholder  equity,  and  with  how 


to  realize  increasing  returns  from  larger-scale  operations  while  not 


necessarily  charging  profit-maximizing  prices. 

Nevertheless,  there  is  a  fundamental  reason  why  the  distinction 
between  market  enterprise  (whether  public  or  private)  and  non-market 
enterprise  is  highly  relevant  and  important  for  understanding  the 
differences  in  governance  (or  management)  in  these  two  types  of 
organizations.  In  market  enterprises,  the  behavior  of  consumers  provides 
invaluable  information  to  management  which  is  not  available  in  non-market 
enterprises.  Consumers  make  millions  of  decisions  and  choices:  whether 
to  spend  or  save;  what  to  buy  and  how  much;  whether  to  buy  product  X  or  Y, 
and,  within  each  product  line,  whether  to  buy  brand  X.^  or  X£.  Moreover, 
these  signals  are  transmitted  frequently  and  regularly,  so  that  comparisons 
over  time  can  be  made.  In  mm,  this  information  relates  closely  to 
sales  and,  by  reflecting  volume,  relates  also  to  costs.  It  affects 
today's  market  shares  and  forecasts  of  tomorrow's,  today's  P&L  "bottom 
line,"  as  well  as  tomorrow's.  It  helps  management  to  make  key  decisions 
concerning  the  enterprise's  employment  levels  and  policies,  salaries  and 
promotions,  internal  budget  allocations,  investment  and  R&D  planning.  The 
point  is  not  that  P&L  directly  affects  all  of  these  decisions,  but  rather 
I  that  it,  and  the  other  aspects  of  P&L  that  result  from  consumer  behavior, 

provide  a  rich  and  invaluable  source  of  information  to  help  guide  these 
«  decisions.  A  market  enterprise  that  pays  insufficient  attention  to  this 

-  * 

V  information  thereby  increases  the  risk  that  it  will  atrophy  and  be  dis- 

*®^,***  placed.  The  process  takes  time,  but  it  is  relentless.  The  experience  of 

»*•'  v  . 

V 

0*  <he  U.S.  automobile  and  steel  industries  is  an  unhappy  reminder  of  this 

.  «/*  fundamental  process. 
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Non-market  enterprises  lack  the  information,  signals,  prods, 
and  guides  that  are  provided  by  the  "bottom  line."  Of  course,  they 
know  their  revenues  and  costs.  But  these  tell  them  little  about  their 
organizations’  performance  because  the  objectives  of  NMEs — their 
raisons  d'etre — are  not  related  to  their  revenues.  Instead,  their 
objectives  typically  relate  to  broad  social  and  public  concerns 
such  as  equity,  aesthetics,  the  advancement  of  knowledge,  science, 
technology,  promotion  of  national  and  international  communication 
and  understanding,  improving  relations  between  government  and  business, 
contributing  to  world  peace,  and  so  on. 

How  then  do  NMEs  conduct  their  operations  in  the  absence  of 
the  usual  type  of  bottom-line  information  available  to  MEs?  What 
internal  norms  or  criteria  or  modes  of  operation  do  they  rely  on 
in  making  certain  types  of  operational  decisions  that  all  organizations 
NME  as  well  as  ME — must  inevitably  make:  for  example,  allocating 
budgets,  evaluating  accomplishments  and  shortcomings,  promoting, 
hiring,  and  terminating  personnel? 


II.  COMMENTS  ON  METHOD 


The  remainder  of  this  paper  will  report  briefly  on  a  preliminary 
and  limited  effort  to  address  these  questions.  The  aim  of  this 
research  has  been  to  understand  how  three  non-market  enterprises  in 
the  same  field  make  certain  types  of  decisions.  The  field  is 
grant-making,  and  the  three  NMEs  include  one  government  organization 
(the  National  Science  Foundation) ,  and  two  non-government  organizations 
(the  Ford  Foundation  and  the  Sloan  Foundation).  We  are  interested 
in  whether  and  what  specific  internal  norms  ("internalities") 
are  used  by  these  organizations  in  making  three  types  of  managerial 
decisions:  those  relating,  respectively,  to  internal  budget  allocation, 
project  evaluation,  and  personnel  evaluation.  Furthermore,  we  are 
interested  in  whether  there  are  discernible  differences  between 
government  and  non-government  NMEs  in  the  way  these  decisions  are  made. 

The  work  I  will  summarize  has  been  principally  conducted  by  a 

graduate  fellow  at  The  Rand  Graduate  Institute,  Charles  Eby, 

* 

working  under  my  supervision.  In  conducting  this  research,  we 
jointly  designed  a  questionnaire  (Attachment  A)  that  was  applied  in 
a  series  of  interviews  held  with  the  chief  executive  officers  of  the 
National  Science  Foundation  and  the  Sloan  Foundation,  and  with  three 
or  four  other  senior  executives  at  those  two  organizations  as  well  as 
at  the  Ford  Foundation.  A  copy  of  the  questionnaire  is  attached  to 
this  paper.  Before  presenting  a  brief  account  of  some  of  the  points 

* 

A  small  grant  to  support  this  research  was  made  in  1930  to  The  Rand 
Graduate  Institute  by  Yale  University's  Program  on  Non-Profit 
Organizations,  directed  by  Professor  John  G.  Simon. 
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and  insights  we  have  learned  in  this  limited  study,  I  will  make  a  few 

* 

observations  on  the  study  s  methodology  and  limitations. 

First,  the  case  study  method  we  have  used,  while  valuable  for 
providing  insights,  does  not  provide  firm  and  reliable  data  for  testing 
hypotheses  in  any  rigorous,  statistical  sense.  Obviously,  this  limitation 
is  even  sharper  when  the  number  of  cases  under  investigation  is  confined  to 

'fc  A 

the  three  foundations  mentioned  above.  In  sum,  the  data  that  can  be 
derived  from  case  studies  are  inherently  "soft,"  often  allowing  ample 
room  for  subjective  interpretations  that  may  vary  for  different 
investigators. 

Second,  the  grant-making  field  under  examination  is  itself  so 
special  that  it  is  quite  legitimate  to  question  whether  any  general 
conclusion  can  be  drawn  from  this  field  with  respect  to  the  broad 
subject  of  "management  without  a  bottom  line"  for  non-market  enterprises 
in  fields  other  than  grant-making. 

Third,  the  views  that  we  elicited  in  the  course  of  our  interviews  were 
at  the  top  executive  levels,  i.e.,  CEOs  or  executive  vice  presidents,  and 
these  views  may  or  may  not  reflect  the  reality  of  management  practices 
at  other  echelons  within  their  organizations.  Nevertheless,  I  should  add 
that  we  were  quite  impressed  by  the  candor,  openness  and  forthcoming  res¬ 
ponses  of  our  interlocutors,  and  by  their  willingness  to  provide 

*** 

documentation  in  support  of  their  observations. 

For  a  fuller  account,  see  Charles  Eby,  "Performance  Norms  in 
Non-Market  Organizations:  An  Exploratory  Survey,"  N-1830, 

The  Rand  Corporation,  April  1982. 

**The  further  dissertation  work  in  which  Eby  is  currently  engaged 
will  expand  the  number  of  cases  by  five  or  six. 

■if 

The  Ford  Foundation  has  been  drastically  reorganized  since  the 
interviews  we  conducted  there  in  the  spring  of  1981,  although  there  was 
quite  clearly  no  cause-effect  relationship  involved  in  this  sequence. 
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III.  PRELIMINARY  FINDINGS 


The  three  grant-making  foundations  differ  in  important  respects 
from  one  another.  The  National  Science  Foundation  was  established 
in  1950  "to  promote  the  progress  of  science;  to  advance  the  national 
health,  prosperity,  and  welfare;  to  secure  the  national  defense; 
and  for  other  purposes."  Its  director  is  appointed  by  the  President 
and  confirmed  by  the  Senate,  as  are  the  twenty-four  part-time  members 
of  the  policy-making  National  Science  Board.  NSF  has  a  staff  of 
about  640  and  an  annual  budget  of  just  under  a  billion  dollars. 

The  broad  purpose  of  the  Ford  Foundation  is  to  contribute 
"to  world  peace  and  to  the  establishment  of  a  world  order  of  law 
and  justice,  ....to  the  basic  principles  of  freedom  and  democracy,  ... 
to  the  economic  well-being  of  people  everywhere,  ...to  conserve  and 
increase  knowledge  and  enrich  our  culture."  Ford  is  the  largest 
private  foundation  in  the  country  with  assets  of  over  two  billion 
dollars,  annual  expenditures  of  about  $160  million,  and  a  staff  of 
nearly  200  at  the  time  of  our  visit  in  March  1981,  although  this  number 
has  been  reduced  since  then  as  a  result  of  the  Foundation's  recent 
reorganization. 

The  Sloan  Foundation  aims  to  "permanently  improve  the  human 

condition. .. [through]  the  steady  growth  of  scientific  and  technical 

knowledge,  and  of  skills  in  managing  the  economy  and  the  public  and 

** 

private  Institutions  of  our  society.  Sloan  has  assets  of  about  a 


From  the  Gaither  Report  of  1950,  as  cited  in  Richard  Magat, 
The  Ford  Foundation,  1979,  p.  18. 

** 

Annual  Report  of  the  Sloan  Foundation,  1979,  p.  5. 


quarter  of  a  billion  dollars,  and  its  annual  grant  expenditures  of 
about  $14  million  are  managed  by  a  staff  of  eleven  professionals. 
How  do  these  different  grant-making  enterprises  manage  their 


affairs  and  make  their  key  decisions  with  respect  to  budgets,  project 
evaluations,  and  personnel  in  the  absense  of  a  profit-and-loss 
bottom-line?  Four  generalizations  seem  to  be  warranted  from  our 
limited  investigation. 

1.  Process  dominates  product 

The  purposes  that  these  non-market  enterprises  seek  to  advance 
are  so  broad,  e.g.,  furthering  the  progress  of  science,  advancing 
national  health,  prosperity  and  welfare,  furthering  the  growth  of 
scientific  and  technical  knowledge,  that  they  cannot  be  realist¬ 
ically  reflected  in  explicit  decisionmaking  criteria.  In  the  absence 
of  such  criteria,  organizational  processes  play  the  predominant  role 
in  decisionmaking.  Decisions  are  developed  by  a  process  that  operates 
from  the  top  down,  and  from  the  bottom  up.  For  example,  the  NSF 
budget  starts  with  a  budget  level  established  by  OMB,  while  the 
Foundation's  programmatic  directorates  develop  their  own  component 
budgets,  which  are  subsequently  reviewed  and  adjudicated  through 

the  director's  office  and  finally  confirmed  by  the  National  Science 

* 

Board.  In  the  organizational  processes  for  evaluating  projects 
and  personnel,  evidence  in  the  form  of  tangible  products  plays  an 
infrequent  role.  Instead,  project  evaluation  ("assessment")  is 
typically  a  peer  review  process  often  involving  outside  panels  of 

*In  the  grant-making  fie  .e  evaluation  is  referred  to 

as  "assessment,"  while  "evaluat . .  i"  occurs  after  the  project  is 
completed. 


peer  reviewers,  while  personnel  evaluation  occurs  through  a  hier¬ 
archical  process  in  which  social  and  communication  skills 
are  often  valued  more  highly  than  tangible  products. 

Of  course,  the  process  of  decisionmaking  in  market  enterprises 
is  also  of  central  importance  in  budget  and  personnel  management. 
However,  in  the  case  of  MEs  this  process  typically  includes,  and 
is  couched  in  terms  of,  tangible  products  or  their  proxies  or 
surrogates,  e.g.,  sales,  profit  margins,  revenues,  etc. 

2.  Lacking  informational  feedback  from  consumer  behavior,  the 
management  of  non-market  enterprises  in  the  grant-making  field 
looks  for  guidance  concerning  priorities  and  activities  to  the 
outside  "marketplace11  of  ideas  and  social  concerns. 

In  establishing,  interpreting  and  adapting  the  broad  purposes 
that  constitute  their  formal  objectives,  NMEs  often  shift  the 
specific  fields  in  which  they  are  engaged  in  accordance  with  the 
vicissitudes  of  intellectual  fashions  and  social  concerns.  One 
example  is  NSF's  current  concern  to  improve  U.S.  industrial 
productivity  and  competitive  positioning  in  world  markets  by 
stimulating  scientific  and  technological  progress.  The  Ford 
Foundation's  program  of  grants  to  encourage  neighborhood  control  of 
primary  schools  in  the  1960s,  and  to  promote  foreign  economic 
development  through  subsidized  technical  assistance  also  in  the  1960s 
and  currently,  are  other  examples.  Ford’s  programmatic  effort  in 
the  environmental  field  in  the  1970s  was  another  example  of  response 
to  intellectual  and  social  ferment  in  the  social  and  intellectual 
"marketplace. " 


3.  Management  of  NMEs  is  affected  by  their  size  and  their  status 


as  government  or  private  enterprises. 

As  the  two  largest  grant-making  organizations.  NSF  and  Ford 

share  certain  characteristics  in  management  style.  Charles  Eby 

describes  the  role  of  the  CEO  at  both  NSF  and  Ford  as  "that  of  a 

monarch — distant  from  the  ordinary  staff  and  attended  by  an  air  of 

* 

formality  and  protocol."  By  contrast,  decisionmaking  at  the  smaller 
Sloan  Foundation  is  collegial  and  shared  among  the  very  small 
number  of  professional  staff  members.  One  consequence  of  this 
difference  is  that  changes  in  programmatic  emphasis  tend  to  be 
associated  with  changes  in  top  management  at  the  two  large  found¬ 
ations,  NSF  and  Ford,  while  programmatic  emphasis  tends  to  be  more 
stable  in  the  face  of  changes  in  top  management  at  Sloan. 

As  a  government  NME,  the  National  Science  Foundation  is 
inevitably  more  subject  to  political  influence  than  are  the  two 
private  foundations.  This  influence  is  exercised  both  within  the 
executive  branch — by  OMB  and  the  White  House — and  by  the  Congress 
through  the  cognizant  Congressional  authorizing  and  appropriation 
committees.  NSF's  recent  decision  to  emphasize  engineering  and 
applied  science  as  contributory  to  U.S.  industrial  productivity,  and 
to  reduce  emphasis  on  the  social  sciences,  is  an  example  of  such 
political  influence.  The  existence  and  importance  of  such  political 
influence  is  not  pejorative.  It  is  simply  a  reflection  of  the  broad 
purpose  assigned  to  its  activities  by  NSF's  authorizing  legislation, 

* 

See  Eby,  pp.  lOff. 
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and  the  consequent  role  of  the  Congress  and  the  Executive  Branch 
in  seeking  to  interpret  and  elaborate  this  broad  social  charter. 

4.  Difficulty  in  managing  NMEs  is  inherent. 

When  allowance  is  made  for  differences  in  size  and  in  the 
government  versus  non-government  status  of  the  grant-making 
organization,  the  management  practices  of  NMEs  in  the  grant-making 
field  seem  generally  to  be  more  similar  than  different.  Management 
without  a  "bottom  line"  is  a  difficult  and  demanding  task  because 
of  the  basic  elusiveness  of  the  broad  social  objectives  that  such 
management  seeks.  Effective  management  of  organizations  that  lack 
a  bottom  line  is  manifestly  more  difficult  precisely  because  of  this 
lack.  Consequently,  decisions  that  are  sensible,  let  alone  "optimal," 
with  respect  to  budgets,  project  activities,  and  personnel  selection 
and  advancement,  are  harder  to  arrive  at  in  non-market  enterprises 
without  a  bottom  line  than  in  market  enterprises  with  it. 

There  is  a  subtle  irony  to  this  situation:  while  the  absence 
of  a  "bottom  line"  makes  good  management  more  difficult  in  non- 
market  enterprises,  this  absence  also  makes  it  more  difficult  to 
demonstrate  the  existence  of  bad  management  in  such  organizations. 
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ATTACHMENT  A 

. 

' 

(Questionnaire) 

INTERNAL  PERFORMANCE  NORMS  IN  NON-MARKET  ORGANIZATIONS 

X! 

-r1 

I.  Organizational  Goals 

;  - 

S 

1.  Is  there  a  formal  statement  of  organizational  goals? 

1.1  What  is  the  source,  and  how  did  it  arise? 

*1 

2 .  Have 
and 

formal  attempts  been  made  to  derive  internal  norms 
standards  from  the  organization's  basic  goals? 

XI 

2.1 

Has  there  been  more  than  one  attempt? 

- 

■ 

2.2 

If  so,  are  they  regular  or  ad  hoc? 

3.  What 

parts  of  the  organization  were  involved  in  these  attempts? 

. 

4.  Are 

these  efforts  viewed  as  successful?  Why? 

--•r 

II.  Budget  Allocation 

1.  What  specific  (or  general)  criteria  (norms,  standards, 
metrics)  are  used  in  allocating  the  foundation's  budget 
among  major  programs,  e.g. ,  basic  or  applied  science. 

• 

1." 

social  sciences,  international  or  domestic,  etc. 

1.1 

To  what  extent  is  continuity  determinative  (e.g., 
this  year's  budget  allocation  depends  mainly  on 
the  budget  share  received  last  year)? 

■ 

1.2 

How  are  changes  in  each  malor  program's  allocation 
decided  upon  (e.g.,  as  a  fixed  proportion  of  the 
change  in  the  foundation's  total  budget,  as  a  result 
of  specific  interventions  by  the  foundation's 
governing  board,  etc.)? 

• 

| 

* 

• 

» 

1.3 

Does  the  foundation  try  to  follow  its  own  precedents 
in  making  allocative  decisions:  rigorously,  frequently, 
or  rarely? 

•  • 

'•  ■ 

r 

1.4 

To  what  extent,  and  in  what  ways,  do  general  political 
considerations  affect  allocative  choices?  Do  these 
considerations  operate  more  directly  and  significantly 
in  NSF 's  decisionmaking  than  in  that  of  the  large 
private  foundations?  Specifically,  how  does  the 
cognizant  Congressional  committee  affect  NSF's  allocative 
choices  among  programs,  and  what  criteria  (or  processes) 
are  reflected  in  this  Congressional  role? 

'  V 

.  * 

• 

'*  ■ 

a 
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1.5  To  what  extent,  in  all  of  the  above  but  especially  in 
(1)  above,  is  it  more  accurate  to  think  of  a  process 
rather  than  a  set  of  criteria  as  determining  allocative 
decisions?  Specifically,  does  a  process  of  subjective 
and  qualitative  judgments,  without  explicit  criteria, 
affect  allocative  decisions  sometimes,  often,  or  usually? 

1.6  To  what  extent  are  the  answers  to  the  above  questions, 
especially  (1),  different  now  from  the  answers  that 
would  have  applied  five  or  ten  years  ago? 

[N.B.  Examples  and  data  on  allocation  decisions  relating  to 
specific  programs  and  patterns  over  time  will  be  particularly 
useful.  If  data  or  reports  exist  that  show,  for  example, 
budget  requests  by  particular  programs,  as  well  as  final 
budget  allocations,  we  would  like  to  see  them.  Explanations 
of  the  differences  between  initial  requests  and  final 
allocations,  e.g.,  in  foundation  records,  or  in  minutes  of 
board  meetings,  would  also  help.] 

III.  Project /Grant  Evaluation 

1.  What  explicit  or  implicit  criteria  (norms,  standards,  metrics) 

are  used  by  the  foundation  in  its  project/grant  evaluations? 

1.1  In  deciding  which  project  options  or  grant  applications 
look  promising  (i.e.,  ex  ante  evaluation)? 

1.2  In  arriving  at  judgments  about  which  projects/grants 
were  highly  successful,  moderately  successful,  or 
disasters  (i.e.,  ex  post  evaluation)? 

1.2.1  Are  the  criteria  used  to  judge  failures  the 

same  as  those  used  to  judge  successes  (i.e.,  are 
the  "scores”  of  failures  simply  the  negative 
values  of  the  same  metrics  used  to  evaluate 
successes)? 

1.2. 1.1  If  not,  why  not? 

1.3  If  the  criteria  referred  to  in  (1.1)  and  (1.2)  are 
different,  why  is  this  so? 

1.4  To  what  extent  are  the  answers  to  (1.1)  and  (1.2) 
different  now  from  what  they  would  have  been  five  or 
ten  years  ago? 
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1.5  How  is  uncertainty  (e.g.,  relating  to  proposed  grants, 
grant  applications,  and  institutions)  dealt  with?  For 
example,  are  some  grant  selections  deliberately  made 
because  they  are  especially  "risky"?  What  sorts  of 
risks  are  accepted  or  sought,  and  what  sorts  are  eschewed, 
e.g.,  with  respect  to  subjects,  people,  or  sponsoring 
institutions? 

[N.B.  Again,  specific  examples  of  "successes"  and  "failures" 
would  be  helpful.] 


IV.  Personnel  Actions 


1.  What  criteria  (norms,  standards,  metrics)  are  used  by  the 

foundation  in  its  regular  staff  decisions? 

1.1  In  hiring  personnel? 

1.2  In  advancing/promoting  personnel? 

1.3  In  providing  regular  and  exceptional  salary  increases? 

1.4  In  providing  regular  and  exceptional  non-monetary 
perquisites? 

1.5  In  arriving  at  judgments  as  to  who  really  are  the  top 
program  officers,  and  who  are  the  poor  performers? 

1.6  In  terminating  personnel? 

1.7  How  different  would  the  answers  have  been  five  or  ten 
years  ago  to  the  preceding  questions  concerning  personnel 
actions? 

[N.B.  Specific  examples  (minus  names)  would  be  helpful.] 


V.  Miscellaneous 

1.  To  what  extent,  and  in  what  ways,  does  the  foundation's  CEO 
affect  the  criteria  and  norms  referred  to  in  the  preceding 
questions? 

1.1  Is  the  CEO's  Influence  persistent  or  occasional,  e.g., 
manifest  when  a  new  CEO  arrives,  but  seldom  thereafter, 
or  regularly  and  continuously? 


-16- 


Attachment  A 
Page  Four 


2.  To  what  extent,  and  In  what  ways,  does  the  foundation's 
governing  board  affect  the  criteria  and  norms  referred  to  in 
the  preceding  questions? 

2.1  Is  the  board's  influence  exercised  collectively,  or 
instead  through  key  individual  members  who  are  especially 
force ful? 

2.2  Is  the  board's  influence  in  these  matters  persistent 
or  occasional? 

3.  Has  decisionmaking  in  the  three  classes  of  operating  decisions 
referred  to  above  become  more  or  less  decentralized  over  time? 

3.1  Why? 

4.  What  salient  difference  (s)  is  found  in  the  criteria  (norms, 
standards,  metrics)  applied  by  other  non-market  organizations 
in  the  grant-making  field? 

4.1  In  government  foundations  vs.  non-government  foundations? 

5.  What  sa  vat  difference  (s)  is  found  in  the  criteria  applied 
by  market  organizations  in  making  the  three  classes  of 
operating  decisions  referred  to  above? 


